Skip to content

fix: cancel ASGI application when a waiting request is cancelled - #5772

Open
DiadkoShmek wants to merge 1 commit into
bentoml:mainfrom
DiadkoShmek:lezo2-fix-asgi-request-cancellation
Open

DiadkoShmek wants to merge 1 commit into
bentoml:mainfrom
DiadkoShmek:lezo2-fix-asgi-request-cancellation

Conversation

@DiadkoShmek

Copy link
Copy Markdown

What does this PR address?

Cancelling a request before its first ASGI response left TimeoutMiddleware's separately scheduled application task running. Catch incoming cancellation while awaiting the first-send waiter, cancel and drain the owned task, then re-raise the request cancellation. A cleanup error must not replace that cancellation.

The regression uses an actual @bentoml.service(traffic={"timeout": 60}), its lifespan and an HTTP POST through httpx.ASGITransport. It fails on the original middleware because handler cleanup has not completed when cancellation returns. After the change the complete local ASGI test module passes (5 tests); additional lifecycle controls cover cooperative delayed cleanup, cleanup errors, successful responses and the existing prompt 504 path.

This handles caller cancellation before response headers. It leaves deadline-triggered and post-header behavior unchanged, and cannot force an application that indefinitely suppresses cancellation to finish. No network-server disconnect or full repository suite is claimed. #5670 addresses a different path: application completion/failure without a first send.

AI assistance: OpenAI Codex investigated, implemented and independently checked this change using local CPU execution. No human code review, provider/model call or upstream acceptance is claimed.

Before submitting:

  • Does the Pull Request follow Conventional Commits naming?
  • Does the code follow BentoML's code style, pre-commit run -a script has passed?
  • Did you read through contribution guidelines and follow development guidelines?
  • Did your changes require updates to the documentation? Have you updated those accordingly?
  • Did you write tests to cover your changes?

The repository-pinned Ruff lint/format checks pass for both changed files, as does git diff --check; the complete pre-commit hook set was not run. This internal cancellation repair does not change the public API.

@DiadkoShmek
DiadkoShmek requested a review from a team as a code owner November 2, 2026 00:37
@DiadkoShmek
DiadkoShmek requested review from bojiang and removed request for a team November 2, 2026 00:37

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant